Mathematical Analysis of Random Noise 

By S. O. RICE 

Introduction 

THIS paper deals with the mathematical analysis of noise obtained by 
passing random noise through physical devices. The random noise 
considered is that which arises from shot effect in vacuum tubes or from 
thermal agitation of electrons in resistors. Our main interest is in the sta- 
tistical properties of such noise and we leave to one side many physical 
results of which Nyquist's law may be given as an example.^ 

About half of the work given here is believed to be new, the bulk of the 
new results appearing in Parts III and IV. In order to provide a suitable 
introduction to these results and also to bring out their relation to the work 
of others, this paper is written as an exposition of the subject indicated in 
the title. 

When a broad band of random noise is applied to some physical device, 
such as an electrical network, the statistical properties of the output are 
often of interest. For example, when the noise is due to shot effect, its 
mean and standard deviations are given by Campbell's theorem (Part I) 
when the physical device is linear. Additional information of this sort 
is given by the (auto) correlation function which is a rough measure of the 
dependence of values of the output separated by a fixed time interval. 

The paper consists of four main parts. The first part is concerned with 
shot effect. The shot effect is important not only in its own right but 
also because it is a typical source of noise. The Fourier series representa- 
tion of a noise current, which is used extensively in the following parts, may 
be obtained from the relatively simple concepts inherent in the shot effect. 

The second part is devoted principally to the fundamental result that the 
power spectrum of a noise current is the Fourier transform of its correlation 
function. This result is used again and again in Parts III and IV. 

A rather thorough discussion of the statistics of random noise currents 
is given in Part III. Probability distributions associated with the maxima 
of the current and the maxima of its envelope are developed. Formulas 
for the expected number of zeros and maxima per second are given, and a 
start is made towards obtaining the probability distribution of the zeros. 

When a noise voltage or a noise voltage plus a signal is applied to a non- 

'An account of this field is given by E. B. Moullin, "Spontaneous Fluctuations of 
Voltage," Oxford (1938). 
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linear device, such as a square-law or linear rectifier, the output will also 
contain noise. The methods which are available for computing the amount 
of noise and its spectral distribution are discussed in Part IV. 
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Summary of Results 

Before proceeding to the main body of the paper, we shall state some of 
the principal results. It is hoped that this summary will give the casual 
reader an over-all view of the material covered and at the same time guide 
the reader who is interested in obtaining some particular item of informa- 
tion to those portions of the paper which may possibly contain it. 

Part I— Shot Effect 

Shot effect noise results from the superposition of a great number of 
disturbances which occur at random. A large class of noise generators 
produce noise in this way. 

Suppose that the arrival of an electron at the anode of the vacuum tube 
at time t = produces an effect F(t) at some point in the output circuit. 
If the output circuit is such that the effects of the various electrons add 
linearly, the total effect at time t due to all the electrons is 

/(') = f> Hi- h) (1.2-1) 

where the fe* electron arrives at tu and the series is assumed to converge. 
Although the terminology suggests that /(/) is a current, and it will be 
spoken of as a noise current, it may be any quantity expressible in the form 
(1.2-1). 

1. Campbell's theorem: The average value of /(/) is 



/(/) = ' I 



F{t) dt ' (1.2-2) 



and the mean square value of the fluctuation about this average is 

ave. [/(/) - lit)? = V f F^(t) dt (1.2-3) 

J- CO 
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where p is the average number of electrons arriving per second at the anode. 
In this expression the electrons are supposed to arrive independently and at 
random, ve^'^ di is the probability that the length of the interval between 
two successive arrivals lies between / and / + dt. 

2. Generalization of Campbell's theorem. Campbell's theorem gives 
information about the average value and the standard deviation of the 
probability distribution of I{t). A generalization of the theorem gives 
information about the third and higher order moments. Let 

/(/) =i:fl.F(^ - k) (1.5-1) 

— 00 

where F{t) and tk are of the same nature as those in (1.2-1) and ■ ■ -ai , 
di , • •• Ok , ■ • • are independent random variables all having the same 
distribution. Then the «*'' semi-invariant of the probabiUty density P(/) 
of / = /(/) is 

Xn= V? r'°[F(i)Tdl (1.5-2) 

J- 00 

The semi-invariants are defined as the coefficients in the expansion of the 
characteristic function /(m): 

loge/(«) = i;^;(^«)" (1-5-3) 

where 



/(«) = ave. e''" = f " P(7)e''" 

J- 00 



-I- "5 

dl 



The moments may be computed from the X's. 

3. As J- — > 00 the probability density P(/) of the shot effect current ap- 
proaches a normal law. The way it is approached is given by 



-1 (0)/..^ ^3*'' <3)i 



P(I) ~ (7 tp {X) — ip {X) 



+ [^%<«(^) + '^' /'(:.)] + 



(1.6-3) 



where the X's are given by (1.5-2) and 



7-7 <.w . 1 rf" - 



^^ = X, x= ' ' ^*"'(r«) = -y^ 

a V27r 



\/27r dx'' 



■x^l2 



Since the X's are of the order of v, a is of the order of v'^ and the orders of 
<r\ \w~\ U(T~'' and Xlff"' are v~"\ v~\ T''' and v"'" respectively. A 



i •- 



•*-,. 
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possible use of this result is to determine whether a noise due to random in- 
dependent events occuring at the rate of v per second may be regarded as 
"random noise" in the sense of this work. 

4. When I(t), as given by (1.5-1), is analyzed as a Fourier series over an 
interval of length T a set of Fourier coefiBcients is obtained. By taking 
many different intervals, all of length T, many sets of coefficients are 
obtained. K i* is sufficiently large these coefficients tend to be distributed 
normally and independently. A discussion of this is given in section 1.7. 

Part II — Power Spectra and Correlation Functions 

1. Suppose we have a curve, such as an oscillogram of a noise current, 
which extends from / = to / = oo. Let this curve be denoted by /(/). 
The correlation function of I(t) is ^(r) which is defined as 

Hr) = Limit J f I{i)Iit + r) dt (2.1-4) 

where the limit is assumed to exist. This function is closely connected 
with another function, the power spectrum, w(f), of I(t). I{t) may be 
regarded as composed of many sinusoidal components. If I(t) were a 
noise current and if it were to flow through a resistance of one ohm the 
average power dissipated by those components whose frequencies lie be- 
tween/and/ 4* (//would be wif) df. 
The relation between w(y) and ^(t) is 

w{f) = 4 /" ^(7) cos IttfTdT (2.1-5) 

Jo 

4,{t) = [ w(f) cos 2irfT df (2.1H3) 

Jo 

When /(/) has no d.c. or periodic components, 

w{f) = Limit ^ ' y^ '' (2.1-3) 

where 

-2T,7i 



SU) = / 

Ja 



I(t)e-''*" dt. 
'0 

The correlation function for 

/(/) = ^ + C co9 (27r/o/ - if) 



IS 



^P{t) = ^' + £ cos 27r/oT (2.2-3) 

These results are discussed in sections 2.1 to 2.4 inclusive. 



. _.r_^-*-^gi:^s^, ■ .^.rirt'iki^i"-' -^*U-«JdL;*.fc.&s-^;«.,'-.. 
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2. So far we have supposed I{t) to be some definite function for which a 
curve may be drawn. Now consider /(/) to be given by a mathematical 
expression into which, besides t, a number of parameters enter. w{J) and 
^(t) are now obtained by averaging the integrals over the possible values 
of the parameters. This is discussed in section 2.5. 

3. The correlation function for the shot effect current of (1.2-1) is 

i^(t) = V j^" F{t)F{t -\-T)dt-\- U j_^ F{t) dt] (2.6-2) 
The distributed portion of the power spectrum is 

where 

sif) = r"^ F(t)e~"'^' dt (2.6-5) 

J- 00 

The complete power spectrum has in_ addition to ivi(f) an impulse func- 
tion representing the d.c. component /(/). 

In the formulas above for the shot effect it was assumed that the expected 
number, v, of electrons per second did not vary with time. A case in which 
V does vary with time is briefly discussed near the end of Section 2.6. 

4. Random telegraph signal. Let 7(0 be equal to either a or -a so that 
it is of the form of a flat top wave, and let the lengths of the tops and bot- 
toms be distributed independently and exponentially. The correlation 
function and power spectrum of / are 

^(r) = a=6-^''' (2.7-4) 

win = 2a2 ' (2.7-5) 

where fx is the expected number of changes of sign per second. 

Another type of random telegraph signal may be formed as follows : Divide 
the time scale into intervals of equal length h. In an interval selected at 
random the value of 1(1) is independent of the value in the other intervals 
and is equally likely to be +a or -a. The correlation function of /(/) is 
zero for I T I > A and is 



••(-'t') 



for < I T I < /! and the power spectrum is 



„(;) = 2h ("-^J (2-7-9) 
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5. There are two representations of a random noise current which are 
especially useful. The first one is 

Hi) = E (an COS wn/ + K sin «„0 (2.8-1) 

n-l 

where an and 5„ are independent random variables which are distributed 
normally about zero with the standard deviation ■\/mj(/„)A/ and where 

0), = 2ir/„ , /„ = mA/ 
The second one is 

J(0 = E '^" cos (w„/ - iprd (2.8hS) 

n-l 

where ^„ is a random phase angle distributed uniformly over the range 
(0, 2ir) and 

c„ = [2wif^)Afr 

At an appropriate point in the analysis N and Af are made to approach 
infinity and zero, respectively, in such a manner that the entire frequency 
band is covered by the summations (which then become integrations). 

6. The normal distribution in several variables and the central limit 
theorem are discussed in sections 2.9 and 2.10. 

Part in — Statistical Properties of Noise Current 

I. The noise current is distributed normally. This has already been 
discussed in section 1.6 for the shot-effect. It is discussed again in section 
3.1 using the concepts introduced in Part II, and the assumption, used 
throughout Part III, that the average value of the noise current I(i) is zero. 
The probabiUty that I{t) lies between I and I -\- dl is 

where ypn is the value of the correlation function, ^{t), of /(/) at r = 

'/'o = lACO) = f wif) dj, (3.1-2) 

Jo 

w{f) being the power spectrum of I{t). i^o is the mean square value of 
/(/), i.e., the r.m.s. value of /(/) is t/'o'^. 
The characteristic function (ch. f.) of this distribution is 

ave.e*"''" = exp-^%' (3.1-6) 



.■','•■'■.. 



- „;i ,.„; ,:; ,:■ -.,_^ 
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2. The probability that /(/) lies between h and 7i + dl, and /(/ + t) 
lies between /a and I2 + dh when / is chosen at random is 

where \pr is the correlation function 1^(7) of /(/): 

^(r) = [ w{f) cos It/t df 



The ch. f. for this distribution is 

iiiliO+ivnt+r) 



ave. e 



= exp — Y («^ + t)^) — lArWW 

3. Tlie expected number of zeros per second of 7(0 is 

rfw{f)df 

Jo 



(3.2-3) 



(3.2-7) 



1 [_rmY = 2 



/ ^(/)tf/ 

JO 



(3.3-11) 



assuming convergence of the integrals. The primes denote differentiation 
with respect to t: 

For an ideal band-pass filter whose pass band extends from/o to fb the ex- 
pected number of zeros per second is 

n/2 
2!;-'^— ^1 (3.3-12) 



L3/5 - /J 



When/o is zero this becomes 1.155/6 and when/a is very nearly equal to 
fb it approaches Jb -\- fa . 

4. The problem of determining the distribution function for the length 
of the interval between two successive zeros of I{i) seems to be quite diffi- 
cult. In section 3.4 some related results are given which lead, in seme 
circumstances, to approximations to the distribution. For example, for 
an ideal narrow band-pass filter the probabiHty that the distance between 
two successive zeros lies between t and t H- rfr is approximately 

dr a 



2 [1 + a\T - T,)1 



213/2 



'-*".. 
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where 



fb -fa 



1 



Tl = 



f»-¥fa 

fb and/o being the upper and lower cut-off frequencies. 

5. In section 3.5 several multiple integrals which occur in the work of 
Part III are discussed. 

6. The distribution of the maxima of /(/) is discussed in section 3.6. The 
expected number of maxima per second is 

j f^^if)df 



21 ^','i 



( fMf)df 



(3.6-6) 



For a band-pass filter the expected number of maxima per second is 

[^fl-AT 



(3.6-7) 



For a low-pass filter where/,, = this number is 0.775 fb . 
The expected number of maxima per second lying above the hne /(/) = /i 
is approximately, when /i is large, 

^-i^nH y i[the expected number of zeros of / per second] (3.6-11) 

where i^-o is the mean square value of /(/). 

For a low-pass filter the probability that a maximum chosen at random 
from the universe of maxima lies between I and I -\- dl is approximately, 
when 7 is large, 



V5^,^-„./= 



dl_ 



(3.6-9) 



where 



7. When we pass noise through a relatively narrow band-pass filter one 
of the most noticeable features of an oscillogram of the output current is 
its fluctuating envelope. In sections 3.7 and 3.8 some statistical properties 
of this envelope, denoted by R or i^(/), are derived. 

The probability that the envelope lies between R and R + dR is 



R ^-R^mo 

^0 



dR 



(3.7-10) 



-v^ ■mi^aiL'rtii'S^- 



Si« ' i) Bt^^ Vul-ufHSiiisyiSSWilimis-iV^:^ JrHr, I 
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where ^o is the mean square value of /(/). The probability that R{t) Ues 
between Rx and Ri -\- dRi and at the same time R(t + r) lies between 
R2 and i?2 + dR2 when / is chosen at random is obtained by multiplying 
(3.7-13) by dRi dR^ . For an ideal band-pass filter, the expected number 
of maxima of the envelope in one second is 

.64n0{/b-/a) (3.8-15) 

When R is large, say y > 2.5 where 

R 

y = --jj2, }pl"^ = r.m.s. value of /(/), 

the probability that a maximum of the envelope, selected at random from 
the universe of such maxima, Hes between R and R -\- dRis approximately 

1.13(^-1).-'"' .4^ 

A curve for the corresponding probability density is shown for the range 
< >> < 4. Curves which compare the distribution function of the maxima 
of R with other distribution functions of the same type are also given. 

8. In section 3.9 some information is given regarding the statistical 
behavior of the random variable : 

E = / l\t) dt (3.9-1) 

where h is chosen at random and I(t) is a noise current with the power 
spectrum w(f) and the correlation function ^(t). The average value 
mr of E is T^o and its standard deviation ct is given by (3.9-9). For a 
relatively narrow band-pass filter 

rrtwf 1 



Mr Vn/b - fa) 

when T(fb - /„) » 1. This follows from equation (3.9-10). An ex- 
pression which is believed to approximate the distribution of E is given by 
(3.9-20). 

9. In section 3.10 the distribution of a noise current plus one or more 
sinusoidal currents is discussed. For example, if / consists of two sine waves 
plus noise : 

I = Pcos pt -j- Q cos qt -f Tk , (3.10-20) 

where p and q are incommensurable and the r.m.s. value of the noise cur- 
rent Zy is ^l'^, the probability density of the envelope R is 

R f rURr)UPr)UQr)€-*''"'Ur (3.10-21) 

where /o( ) is a Bessel function. 
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Curves showing the probability density and distribution function of R, 
when Q = 0, for various ratios of P/i.m.s. In are given. 

10. In section 3.11 it is pointed out that the representations (2.8-1) 
and (2.8-6) of the noise current as the sum of a great number of sinusoidal 
components are not the only ones which may be used in deriving the results 
given in the preceding sections of Part III. The shot effect representation 

m = i; ^e - '*) 

—00 

studied in Part I may also be used. 

Part IV — Noise Through Non-Linear Devices 

1. Suppose that the power spectrum of the voltage V applied to the 
square-law device 

I = aV' (4.1-1) 

is confined to a relatively narrow band. The total low-frequency output 
current lu may be expressed as the sum 

/,/ = U. + /// ' (4.1-2) 

where /dc is the d.c. component and !(/ is the variable component. When 
none of the low-frequency band is eliminated (by audio frequency filters) 

lU = "-^ (4.1-6) 

where R is the envelope of V. If V is of the form 

V = Vy -\- P cos pi -\- Q cos gt, (4.1-4) 

where Vn is a noise voltage whose mean square value is ^o , then 

/^ = a^ [^l + I^h + QVo + ^'] (4.1-16) 

2. If instead of a square-law device we have a linear rectifier, 

/ = /° ^<° (42-1) 

^ \aV, V>Q ^ ^ 



the total low-frequency output is 



ia=~ (4.2-2) 



'•.■.VjfcW.^-S :.-. 
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When F is a sine wave plus noise, Fy + Pcos pt. 



he = «(0"i^i(-^; 1; -*) (4-2-3) 

• /S=4(^' + 2^o) (4.2-6) 

where \Fi is a hypergeometric function and 

_ i^ _ Ave. sine wave power . ., 

2i^o Ave. noise power 

When X is large 

If V consists of two sine waves plus noise, Idc consists of a hypergeometric 
function of two variables. The equations running from {A.1-9) to (4.2-lS) 
are concerned with this case. About the only simple equation is 

/S == ^ [2^0 •\-P'-\- Q'] (4.2-14) 

IT 

3. The expressions (4.1-6) and (4.2-2) for lit in terms of the envelope 
R of V, namely 

-2. -d ~, 

are special cases of a more general result 

Id = Ao(R) =~ ( F{iu)Jo{uR) du. (4.3-11) 

In this expression JoiuR) is a Bessel function. The path of integration C 
and the function F(iu) are chosen so that the relation between / and V may 
be expressed as 

/ = A /■ F{iu)e'^'' du. (4A-1) 

2t Jc 

A table giving F(iu) and C for a number of common non-linear devices is 
shown in Appendix 4A. 

If this relation is used to study the biased linear rectifier. 

/ 0, V <B 

\V - B, V > B 



..^^K.^WiJ^KlifiWi 
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for the case in which V is Vn -\- P cos pt, we find 

2 IT 2ttP 

_ „ , (4.3-17) 

when P ;S> I JB I , /* ^ f o where V'o is the mean square value of Vn . 

4. When V is confined to a relatively narrow band and there are no 
audio-frequency filters, the probabiUty density and all the associated sta- 
tistical properties of la may be obtained by expressing Id as a function 
of the envelope i? of K and then using the probability density of R. When 
V is Fjv + -P cos pt + Q cos qt this probability density is given by the in- 
tegral, (3.10-21) (which is the integral containing three Bessel functions 
stated in the above summary of Part III). When V consists of three sine ' 
waves plusjioiEe there are four A's in the integrand, and so on. Expres- 
sions for R" when R has the above distribution are given by equations 
(3.10-25) and (3.10-27). 

When audio-frequency filters remove part of the low-frequency band the 
statistical properties, except the mean square value, of the resulting cur- 
rent are hard to compute. In section 4.3 it is shown that as the output band 
is chosen narrower and narrower, the statistical properties of the output 
current approach those of a random noise current. 

5. The sections in Part IV from 4.4 onward are concerned with the 
problem: Given a non-linear device and an input voltage consisting of noise 
alone or of a signal plus noise. What is the power spectrum of the output? 
A survey of the methods available for the solution of this problem is given 
in section 4.4. 

6. When a noise voltage I'V with the power spectrum w(/) is appHed to 
the square-law device 

/ = aV'' (4.1^1) 

the power spectrum of the output current / is, when/ ?>£ 0, 

/+00 
w{x)wij ~ x)dx (4.5-5) 

00 

where wi—x) is defined to equal w{x). The power spectrum of / when V 
is either P cos pt -f Vn or 

Q{\ -f k cos pt) cos qt -f- Vn 

is considered in the portion of section 4.5 containing equations (4.5-10) to 
(4.5-17). 



Ml ti ii mi*^ umb m J ^at.t ^ma^ i f% ii: ^ ^itUiU^':i^^t«iiS^-iAt^ • ■ Tife.-=t=:^s--,uK--<ii»^^.>iMMw:^ tiAi^ 
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7. A method discovered independently by Van Vleck and North shows 
that the correlation function *(t) of the output current for an unbiased 
linear rectifier is 

^„=| + ^.^[-J,_i;^;g (4.7-6) 

where the input voltage is Vn . The correlation function ^(t) of Vy is 
denoted by ^r and the mean square value of Vn is ^o . The power spectrum 
Wif) of / may be obtained from 

WU) = 4 [ *(t) cos 2vfT dr (4.6-1) 

by expanding the hypergeometric function and integrating termwise using 

G„(/) = [ ^; cos 2t/t dr. (4C-1) 

JQ 

Appendix 4C is devoted to the problem of evaluating the integral for G„(/). 

8. Another method of obtaining the correlation function i/'(t) of /, termed 
the "characteristic function method," is explained in section 4.8. It is 
illustrated in section 4.9 where formulas for *(t) and W(J) are developed 
when the voltage P cos pt + Vn is applied to a general non-Unear device. 

9. Several miscellaneous results are given in section 4.10. The char- 
acteristic function method is used to obtain the correlation function for a 
square-law device. The general formulas of section 4.9 are applied to the 
case of a v^ law rectifier when the input noise spectrum has a normal law 
distribution. Some remarks are also made concerning the audio-frequency 
output of a Unear rectifier when the input voltage V is 

Q{1 -\- r cos pt) cos gt-\- Vn. 

10. A discussion of the hypergeometric function iFi{a; c; x), which often 
occurs in problems concerning a sine wave plus noise, is given in 
Appendix 4B. 

PART I 

THE SHOT EFFECT 

The shot effect in vacuum tubes is a typical example of noise. It is due 
to fluctuations in the intensity of the stream of electrons flowing from the 
cathode to the anode. Here we analyze a simplified form of the shot effect. 
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1.1 The Probability of Exactly K Electrons Arriving at the 
Anode in Time T 

The fluctuations in the electron stream are supposed to be random. We 
shall treat this randomness as follows. We count the number of electrons 
flowing in a long interval of time T measured in seconds. Suppose there 
are A"i . Repeating this counting process for many intervals all of length 
T gives a set of numbers Kz , K3 • • • Km where M is the total number of 
intervals. The average number »-, of electrons per second is defined as 

tt-*ta Mi 

where we assume that this limit exists. As M is increased with T being 
held fixed some of the K's will have the same value. In fact, as M increases 
the number of it's having any particular value will tend to increase. This 
of course is based on the assumption that the electron stream is a steady 
flow upon which random fluctuations are superposed. The probability of 
getting K electrons in a given trial is defined as 

,„, _ _ . Number of trials giving exactly K electrons , . 

jtf-toe M 

Of course p{K) also depends upon T. We assume that the random- 
ness of the electron stream is such than the probability that an electron 
will arrive at the anode in the interval (/, t + A/) is vM where A/ is 
such that vM « 1, and that this probabiUty is independent of what has 
happened before time / or will happen after time / + A/. 

This assumption is sufficient to determine the expression for p{K) which is 

pw = ^4^'"" ^^-^'^^ 

This is the "law of small probabilities" given by Poisson. One method 

of derivation sometimes used can be readily illustrated for the case A^ = 0. 

T 
Thus, divide the interval, (0, T) into M intervals each of length ^ = rrz.. 

M 

A/ is taken so small that viid is much less than unity. (This is the "small 
probability" that an electron will arrive in the interval A/). The prob- 
ability that an electron will not arrive in the first sub-interval is (1 — v^t). 
The probability that one will not arrive in either the first or the second 
sub-interval is (1 — v^t) . The probability that an electron will not arrive 
in any of the M intervals is (1 — vdJ)", Replacing M by T/M and letting 
A/ ^ gives 
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The expressions for ^(1), p(2), • ■ • p{K) may be derived in a somewhat 
similar fashion. 

1.2 Statement of Campbell's Theorem 
Suppose that the arrival of an electron at the anode at time / = produces 
an effect F(t) at some point in the output circuit. If the output circuit 
is such that the effects of the various electrons add linearly, the total effect 
at time / due to all the electrons is 

m = T, Fit- fk) (1.2-1) 

where thfe k^^ electron arrives at th and the series is assumed to converge. 
Campbell's theorem^ states that the average value of I{t) is 

7(0 = .' f'^ F{t)di (1.2-2) 

J- BO 

and the mean square value of the fluctuation about this average is 

J— 00 

where v is the average number of electrons arriving per second. 

The statement of the theorem is not precise until we define what we mean 
by "average". From the form of the equations the reader might be tempted 
to think of a time average; e.g. the value 

Limi f I{t)dt (1.2-4) 

3-.00 i Jo 

However, in the proof of the theorem the average is generally taken over 
a great many intervals of length T with t held constant. The process is 
somewhat similar to that employed in (1.1) and in order to make it clear 
we take the case of I{t) for illustration. We observe I{t) fcr many, say M^ 
intervals each of length T where T is large in comparison with the interval 
over which the effect F{t) of the arrival of a single electron is appreciable. 
Let J{t') be the value of /(/), t' seconds after the beginning of the w* in- 
terval. /' is equal to / plus a constant depending upon the beginning time 
of the interval. We put the subscript in front because we wis h to reserve 
the usual place for another subscript later on. The value of I[t') is then 
defined as 

7(0 = Limit i y{t') + a{t') + - • ■ + Ml{t')] (1.2-5) 

and this limit is assumed to exist. The mean square value of the fluctua- 
tion of /(/') is defined in much the same way. 

* Ptoc. Camh. Phil. Soc. IS (1909), 117-136, 310-328. Our proof is similar to one given 
by J. M. Whittaker, Proc. Camb. Phil. Soc. 33 (1937), 451-458. 
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Actually, as the equations (1.2-2) and (1.2-3) of Campbell's theorem 
show, these averages and all the similar averages encountered later turn 
out to be independent of the time. When this is true and when the M in- 
tervals in (1.2-5) are taken consecutively the time average (1.2-4) and the 
average (1.2-5) become the same. To show this we multiply both sides of 
(1.2-5) by dt' and integrate from to T: 

W) = Lmit ^T^llf m/(/0 dl' 

, .ur (1-2-6) 

= Limit -~ / /(O di 

.U-a) Ml Jo 

and this is the same as the time average (1.2-4) if the latter limit exists. 

1.3 Proof of Campbell's Theorem 

Consider the case in which exactly K electrons arrive at the anode in an 
interval of length T. Before the interval starts, we think of these K elec- 
trons as fated to arrive in the interval (0, T) but any particular electron is 
just as likely to arrive at one time as any other time. We shall number 
these fated electrons from one to K for purposes of identification but it is to 
be emphasized that the numbering has nothing to do with the order of ar- 
rival. Thus, if ik be the time of arrival of electron number k, the probability 
that tk lies in the interval (i, i -|- dl) is dt/T. 

We take T to be very large compared with the range of values of / for 
which F(t) is appreciably different from zero. In physical appHcations 
such a range usually exists and we shall call it A even though it is not very 
definite. Then, when exactly K electrons arrive in the interval (0, T) the 
effect is approximately 

K 

!kU) = 'EF(t- k) (1.3-1) 

k-i 

the degree of approximation being very good over all of the interval except 
within A of the end points. 

Suppose we examine a large number M of intervals of length T. The 
number having exactly K arrivals will be, to a first approximation M p{K) 
where p{K) is given by (1.1-3). For a fixed value of t and for each interval 
having K arrivals, /k(/) will have a definite value. As Af ^ oo , the average 
value of the IkW^, obtained by averaging over the intervals, is 

Jo I Jo 1 k-^l 

= E/ tF(/-4) 

k-i Jo I 



■rtlMll1'';imVK^^''^l^rl■^n^k^»■l^f1^^"fl*1flfafi*''T^'r^^hfV ' - r.--iiPsttMu^i^-^^^'KfC:iiSi'4^iiAuk .^..u.ii^dii.:-^''-*^ •---'- 
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and if A < / < r — A, we have effectively 

7:(^)^f[y(i)di (1.3-3) 

If we now average I(t) over all of the M intervals instead of only over 
those having K arrivals, we get, as M" — * oo , 

7(0 = E P(K)T:iJ) 



frkT Kl 1« ^^ 



= V [ " F{t) dt (1.3-4) 

J- 00 

and this proves the first part of the theorem. We have used this rather 
elaborate proof to prove the relatively simple (1.3-4) in order to illustrate a 
method which may be used to prove more complicated results. Of course, 
(1.3-4) could be established by noting that the integral is the average value 
of the effect produced by one arrival, the average being taken over one 
second, and that v is the average number of arrivals per second. 

In order to prove the second part, (1.2-3) of Campbell's theorem we first 
compute P{t) and use 

(/(o - m? = iHi)-2 mm + Wf 

= PU)~ Wf (1-3-s) 

From the definition (1.3-1) of /«(/), 

lUi) = E E ^(' - iicWt - O 

Averaging this over all values oihjz, ■••Ik with t held fixed as in (1.3-2), 
The multiple integral has two different values. If * = m its value is 
and if A 7^ m its value is 

r-T J, rT 



r..-.)ff.(.-c)^" 
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Counting up the number of terms in the double sum shows that there are K 
of them having the first value and K'^ —A" having the second value. Hence, 
if A < / < r - A we have 

/Iw = f /7 ^^'^ '^^ + ^^^^ [C ^^'^ ^' 

Averaging over all the intervals instead of only those having K arrivals 
gives 



J- 00 



where the summation with respect to A' is performed as in (1.3-4), and after 
summation the value (1.3-4) for /(/) is used. Comparison with (1.3-5) 
establishes the second part of Campbell's theorem. 

1.4 The Distribution of I{t) 

When certain conditions are satisfied the proportion of time which /(/) 
spends in the range I, I -\- dl is P(I)dI where, as v ^ oo , the probability 
density P{I) approaches 

^ .-"-^'^'-1 (1.4-1) 



where /is the average of /(/) given by (1.2-2) and the square of the standard 
deviation o-j, i.e. the variance of I{t), is given by (1.2-3). This normal 
distribution is the one which would be expected by virtue of the "central 
limit theorem" in probability. This states that, under suitable conditions, 
the distribution of the sum of a large number of random variables tends 
toward a normal distribution whose variance is the sum of the variances 
of the individual variables. Similarly the average of the normal distribu- 
tion is the sum of the averages of the individual variables. 

So far, we have been speaking of the limiting form of the probabiHty 
density P{I). It is possible to write down an explicit expression for P{I), 
which, however, is quite involved. From this expression the limiting form 
may be obtained. We now obtain this expression. In line with the dis- 
cussion given of Campbell's theorem, we seek the probability density P(I) 
of the values of /(/} observed at / seconds from the beginning of each of a 
large number, M, of intervals, each of length T. 



thu i-^H wij" 
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Probability that 7(0 lies in range (7, 1 + dl) 

= ^ (Probability of exactly iC arrivals) X 

(Probability that if there are exactly 
A' arrivals, Uit) lies in (7, 7 + dl)). 
Denoting the last probability in the summation by PK{I)dI, using notation 
introduced earlier, and cancelling out the factor dl gives 

P{I) = f: PWPkU) (1.4-2) 

We shall compute Pk{I) by the method of "characteristic functions"' from 
the definition 



K 



lK(t) = TF{t-h) (1.3-1) 

of I Kit). The method will be used in its simplest form: the probability that 
the sum 

Xi -\- xz -\- ■ ■ • ■{■ Xk 
of K independent random variables lies between X and X + rfX is 

dX— I " e-''"' n (average value of e"'") du (1.4-3) 

27r J-ao k=l 

The average value of e''"", i.e., the characteristic function of the distribution 
of Xk , is obtained by averaging over the values of Xk . Although this is the 
simplest form of the method it is also the least general in that the integral 
does not converge for some important cases. The distribution which gives 
a probability of h that Xk = - 1 and ^ that xt = ^-1 is an example of such a 
case. However, we may still use (1.4r-3) formally in such cases by employ- 
ing the relation 



L 



e-""'du = 27r5(a) (1.4-4) 



where 5(a) is zero except at a = where it is infinite and its integral from 
a = — etoa= +eis unity where e > 0. 
When we identify Xk with F{t - h) we see that the average value of 



IfcU - 

e IS 



1 f^ 

i / exp [iuFit - tk)] dtk 
T Ja 



' The essentials of this method are due to Laplace. A few remarks on its history are 
given by E. C. Molina, Bull. Amsr. Math. Soc, 36 (1930), pp. 369-392. An account of 
the method may be found in any one of several texts on probability theory. We mention 
"Random Variables and Probabihty Distributians," by H. CramSr, Camb. Tract m 
Math and Math, Phys. No. 36 (1937), Chap. TV. Also "Introduction to Matheraaticai 
Probability," by J. V. Uspensky, McGraw-HiU (1937), pages 240, 264, and 271-278. 
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All of the K characteristic functions are the same and hence, from (1.4-3), 
PK(I)dI is 

Although in deriving this relation we have taken iC > 0, it also holds for 
K = (provided we use (1.4r4)). In this case Po{T) = 5(1), because 7=0 
when no electrons arrive. 

Inserting our expression for Pk(I) and the expression (1.1-3) for p(K) 
in (1.4-2) and performing the summation gives 

-\- V J exp [iuF(t - t)] dTJdu (1.4-5) 

The first exponential may be simplified somewhat. Using 

vT = V I dr 

permits us to write 

-vT + V \ exp {iuF{t - t)] rfr = v / (exp [iiiF{t - t)] - 1) dr 
Jo JO 

Suppose that A < / < T — A where A is the range discussed in connection 
with equation (1.3-1). Taking | F(/ - t) | = f or [ / - t | > A then 
enables us to write the last expression as 



u 



f [e'"""' ^l]dt (1.4-6) 

Placing this in (1.4-5) yields the required expression for P(T): 

W) = ^ / exp (^-ilu + V j^ [e*"^'" - 1] dt\dii (1.4-7) 

An idea of the conditions under which the normal law (1.4-1) is ap- 
proached may be obtained from (1.4-7) by expanding (1.4-6) in powers of 
u and determining when the terms involving «^ and higher powers of « 
may be neglected. This is taken up for a slightly more general form of 
current in section 1.6. 
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1.5 Extension of Campbell's Theorem 

In section 1.2 we have stated Campbell's theorem. Here we shall give 
an extension of it. In place of the expression (1.2-1) for the /(/) of the shot 
effect we shall deal with the current 

7(0 = E a,F(t--t,) (1.5-1) 

where F{t) is the same sort of function as before and where • • ■ ai , (h , ••■ 
at , - ■ ■ are independent random variables all having the same distribution. 
It is assumed that all of the moments a" exist, and that the events occur at 

random 

The extension states that the «th semi-invariant of the probability density 
P{I) of /, where I is given by (1.5-1), is 

\.= va- I mWdt (1.5-2) 

where v is the expected number of events per second. The semi-invariants 
of a distribution are defined as the coefficients in the expansion 

log. (ave. ."") = i: ^ {iuT + oiu^) (1.5-3) 

n— 1 Ml 

i.e. as the coefficients in the expansion of the logarithm of the characteristic 
function. The X's are related to the moments of the distribution. Thus if 
mi , ma , ■ ■ ■ denote the first, second • • • moments about zero we have 

ave. ."" = 1 + i: -: (*'«)" + "(«") 

n-l Ml 

By combining this relation with the one defining the X's it may be shown that 

/ = mi = Xi 

/2 = ma = Xa + XiWi 

73 = wa = Xs -f 2X2mi + Xima 



It follows that Xi = / and Xa = ave. (/ - if. Hence (1.5-2) yields the 
original statement of Campbell's theorem when we set n equal to one and 
two and also take all the o's to be unity. 

The extension follows almost at once from the generalization of expression 
(1.4^7) for the probabiUty density P{I). By proceeding as in section 1.4 
and identifying Xk with akF{t - tk) we see that 

ave. e"*" = i f q{a) da [ exp [iuaF{t - Q] dh 
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where qia) is the probability density function for the a's. It turns out that 
the probability density P(I) of / as defined by (1.5-1) is 

P{r) = ^ /^ exp (-ilu + ,; jf^ q{a) da 
The logarithm of the characteristic function of P(l) is, from (1.5-4), 

V 



[ q{a)da [ [e"""'" - l]di 

n=l «I J-00 J-w 



(f^ 



Comparison with the series (1.5-3) defining the semi-invariants gives the 
extension of Campbell's theorem stated by (1.5-2). 

Other extensions of Campbell's theorem may be made. For example, 
suppose in the expression (1.5-1) for I{t) that h, h , • • ■ tk, ■ ■ • while still 
random variables, are no longer necessarily distributed according to the 
laws assumed above. Suppose now that the probability density p(x) is 
given where x is the interval between two successive events: 

h = h-\-xi (1.5-5) 

ii = h -\- x^ = ti -h xi -\- X3 

and so on. For the case treated above 

p(x) = ve-"\ (1.5-6) 

We assume that the expected number of events per second is still v. 
Also we take the special, but important, case for which 

F{t) = 0, t <0 " (1.5-7) 

F(t) = e-°\ t > 0. 

For a very long interval extending from t = ti to t = T + ^1 inside of which 
there are exactly K events we have, if t is not near the ends of the interval, 

/(/) = aiF(t - h) + a^FQ. - h - x,) -\ 

+ ajc+i-P(^ — h— xi--- ~ Xk) 

«= axF{t') + o*F(/' - «i) + ■ • ■ + a^+iF(/' - xi Xk) 



■i^*i-^-i-M-X ___-irhi«»Ui.= r-7V---TtffW:'j».- 
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where q(a) is the probability density function for the a's. It turns out that 
the probability density P(I) of / as defined by (1.5-1) is 

P{I) = -z- I exp ( —ilu -\- V I q(a) da 

2t J-bo \ J-aa 

j^""" [g-af(« _ l]jA(^„ (1.5-4) 

The logarithm of the characteristic function of P{2) is, from (1.5-4), 
V [ " g(a)da ( [e*""''"' - l]dt 






dt 



Comparison with the series (1.5-3) defining the semi-invariants gives the 
extension of Campbell's theorem stated by (1.5-2). 

Other extensions of Campbell's theorem may be made. For example, 
suppose in the expression (1.5-1) for /(/) that h,h, ■ ■ ■ h, ■ • ■ while still 
random variables, are no longer necessarily distributed according to the 
laws assumed above. Suppose now that the probability density p(x) is 
given where x is the interval between two successive events : 

h = h-\- xi (1.5-5) 

h = h-h X2 = h-h Xi-\- X2 

and so on. For the case treated above 

p(x) = ve~". (1.5-6) 

We assume that the expected number of events per second is still v. 
Also we take the special, but important, case for which 

F(t) =0, i <0 (1.5-7) 

F(t) = e~°\ t > 0. 

For a very long interval extending from t = titot = T -^ ti inside of which 
there are exactly K events we have, if t is not near the ends of the interval, 

/(/) = aiF{t - h) + azFit - h - x{) ■] 

+ aK+iF(t — h — Xi ■ • • — Xk) 

= aiF{t') + asFit' - xi) -\ + aK+iF(t' - xi x^) 



_--.-'j^ '■ji^^P^.^-j^ ^^ *^- *g W t * i". i ^ -Pj^-V* -^-' ^^ .-_',/ -t ^r- * 
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I\l) = alF\t') + a\F\t' - r^i) + • • ■ + aUiP\i' - Xi xk) 

+ 2aia2F(i')F{l' - xi) -\ + 2aiaK+iF(t')F{t' - xi Xk) 

+ 2a2azF{t' - Xi)F{t' - xi - X2) -\ + ■ • ■ 

where t' = t — ti. If we integrate /\7) over the entire interval < i' < T 
and drop the primes we get approximately 



/ I\t)dt = (ol + ■ • ■ + aUi)<p{G) 
Ja 



-\- 2aia2tpixi) + 2aia3(p(xi + .r2) + ■ ■ • + 2aiaii:4.i^(a;i + • ■ • + ^k) 
4- 20203^(3^2) + ••• + ■■■ 4- 2aKaK^itp(xK) 
where 

^(x) = I F(i)F{t - x) d% 
When we divide both sides by T and consider K and T to be very large, 
f °'+"^+°'" >>(0)'=mV(0) 

1 TT 

r;[oia2^Ca:i) +^a3vC*2) + ■ ■ ■ +«kOk+iv'(*k)I = ^ average 0^0^+1 ^fyO ■ 



= v(^ I <f(x)p(x) dx 
Jo 



1 TT — 1 

^ [aia^tpixi -\- X2) + ••■] = — p— ave. akak+$tp(xk + Xk+i) 

= vd^ j dxi I dx2p(xi)p(x'i)>fixi + Xi) 
Jo Jo 

Consequently 

l rT 

P{t) =Lim- f(t)di 

T-teo I Jo 

= mVCO) + 2va^ J p{x)ip{x) dx 

+ / dxi \ dx2p(xi)p{x2)(p(xi + X2) + 
Jo J<i 

For our special exponential form (1.5-7) for F{i), 

^(«) = 2^ 
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and the multiple integrals occurring in the expression for P{l) may be written 
in terms of powers of 

= [ p{x)e-"dx (1.5-8) 



Thus 



and since 



we have 



' .0 



2ar-{l) = va? + 2a V ^ 



1-9 



F{t) at = va/a 



m-m'-i + {;^)'[~^,-q a-^-") 



2a 



Equations (1.5-8) and (1.5-9) give us an extension of Campbell's theorem 
subject to the restrictions discussed in connection with equations (1.5-5) 
and (1.5-7). Other generalizations have been made* but we shall leave the 
subject here. The reader may find it interesting to verify that (1.5-9) 
gives the correct answer when p{x) is given by (1.5-6), and also to investi- 
gate the case when the events are spaced equally. 

1.6 Approach or DisTRiBrTioN ov I to a Normal Law 

In section 1.5 we saw that the probability density P{1) of the noise current 
/ may be expressed formally as 

P(/) + J_ r^ cxp\-iTu + f: (iuy\Jn\\du (1.6-1) 
2ir J-,0 L "^^ -1 

where \n is the «th semi-invariant given by (1.5-2). By setting 

Ai = tr 
7 - X, 7-7 



X — 



(1.6-2) 



J See E. N. Rowland, Proc. Camb. Phil. Soc. 32 (1936), 580-597. He extends the 
theorem to the case where there are two functions instead of a single one, which we here 
denote by lit). According to a review in the Zentralblatt fur Math., 19, p. 224, Khint- 
chine in the Bull. Acad. Sci. URSS, s;r. Math. Nr. 3 (1938), 313-322, has continued and 
made precise the earlier work of Rowland. 



.. -i^-**i--_Jr<-- -..*«6i>'ii"=i; ■I'ft*^**-""'*"'- -*>fci'^ I'-v*!'*" : . 



-S ' ■ '.'■;'•' 
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expanding , 

exp i; {iu)"\../v\ 
as a power series in w, integrating termwise using 

and finally collecting terms according to their order in powers of v~^'^, gives 

n/T\ -1 Wf \ XaO" (3)/ \ [ I Xiff (4)/ , , X3O" (6)/ \ I , 

(1.6-3) 

The first term is f)(v~^ ), the second term is 0(f~^), and the term within 
brackets is Oiv"^ '). This is Edgeworth's series. The first term gives the 
normal distribution and the remaining terms show how this distribution is 
approached as j' — * =» . 

1.7 The Fourier Components of I(i) 
In some analytical work noise current is represented as 

HO = "2 + 2^^ (a. cos — + i„ sm -^1 (1./-1) 

where at a suitable place in the work Tand jVare allowed to become infinite. 
The coefficients a„ and bn , I < n < N, are regarded as independent random 
variables distributed about zero according to a normal law.. 

It appears that the association of (1.7-1) with a sequence of disturbances 
occurring at random goes back many years. Rayleigh^ and Gouy suggested 
that black-body radiation and white light might both he regarded as se- 
quences of irregularly distributed impulses. 

Einstein and von Laue have discussed the normal distribution of the 
coefficients in (1.7-1) when it is used to represent black-body radiation, this 
radiation being the resultant produced by a great many independent os- 

' See, [or example, pp. 86-87, in "Random Variables and Probability Distributions" 

bv H. Cramer, Cambridge Tract No. 36 (1937). 
" ''Phil. Mag. Ser. 5, Vol. 27 (1SS9) pp. -160-469. 
I A. Einstein and L. Hnpf, Ann. d. Pkvsik 3i (1910) pp. 1095-1115. 
■ M. V. Laue, Ann. d. Physik 47 (1915) pp. 853-878. 
A. Einstein, Ann. d. Physik 47 (1915) pp. 879-885. 
M. V. Laue, A7m. d. Physik 48 (1915) pp. 668-680. 
I am indebted to Prof. Goudsmit for these references. 



-^ 
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cillators. Some argument arose as to whether the coefficients in (1.7-1) 
were statistically independent or not. It was finally decided that they 
are independent. 

The shot effect current has been represented in this way by Schottky. 
The Fourier series representation has been discussed by H. Nyquist^ and 
also by Goudsmit and Weiss. Remarks made by A. Schuster'" are equiv- 
alent to the statement that a„ and b^ are distributed normally. 

In view of this wealth of information on the subject it may appear super- 
fluous to say anything about it. However, for the sake of completeness, 
we shall outUne the thoughts which lead to (1.7-1). 

In line with our usual approach to the shot effect, we suppose that exactly 
A' electrons arrive during the interval (0, T), so that the noise current for 
the interval is 

IkU) = E F(/ - h) (1.7-2) 

1=1 

The coefficients in the Fourier series expansion of 7^(0 over the interval 
(0, 7") are a„K and bnK where 



r . 2wnn 



dt 



2 \ 

QnK - ibnK = ^ £ / F(i - /fc) BXp 

T t=i Jo 

-||:£"f(oe..p[-i'-^^/ + /.)]rf/ 
= i?„fi-"'^"i: «-*"'"■■ (1.7-3) 

In this expression 



q - 27r/t 



-= C„ ~iS. = j,[^ me-'""" 



(1.7-4) 



H.e-'"- = C„ - 75. = ^ F(/)f-'^""' dt 



In the earlier sections the arrival times h , t2 , ■ ■ ■ Ik were regarded as A' 
indejiendent random variable each distributed uniformly over the interval 
(0, T). Hence the 5a's may be regarded as random variables distributed 
uniformly over the interval to 2r. 

Incidentally, it will be noted that in (1.7-3) there occurs the sum of A 
randomly oriented unit vectors. When K becomes very large, as it does 

»Aiiii. d. Physik, 57 (191S] pp. 541-567. 

^ Unpublished Memoratuium, "Fluctuations in Vacuum Tube Noise and the Like," 
March 17, 1932. 

^"Investigation of Hidden Periodicities, Terrestrial Magnetism, 3 (1898), pp. 13-41. 
See especially propositions 1 and 2 on page 26 of Schuster's paper. 



■oWi^'Wtftfe • M«rM i MHt iti^>^S^'.fT ^''-a ti £ , .tii t i j ' iaiaiii ii tir f ^ MTra ^WMi^»U*A^Mi«<;^^MWi<i«i ^ r ^-.^^^ *■ 



308 BELL SYSTEM TECHNICAL JOURNAL 

when c — * «> , it is known that the real and in^aginary parts of this sum are 
random variables, which tend to become independent and normally dis- 
tributed about zero. This suggests the manner in which the normal dis- 
tribution of the coefficients arises. Averaging over the fit's in (1.7-3) gives 
when w > 

d.K = 5„^ = (1.7-5) 

Some further algebra gives 

flnK = 0„K ~ — Kn 

2 (1.7-6) 



where n 9^ m and n,m > 0. 

So far, we have been considering the case of exactly K arrivals in our 
interval of length T. Now we pass to the general case of any number of 
arrivals by making use of formulas analogous to 

A-O 

as has been done in section 1.3. Thus, for « > 0, 

On = 5n = 

"T TJ VT ri2 2 /I - o\ 

a,. = ft„ = Y i?n = ff„ (1.7-8) 

a^bn. = a,^a,n = h^b^ = 0, 11 ?^ m 

In the second hne we have used a„ to denote the standard deviation of an 
and bn ■ We may put the expression for fl-„ in a somewhat different form 
by writing 

/. = ^ = nAf, A/ = ^ (1.7-9) 

where /„ is the frequency of the wth component. Using (1.7-4), 



2 

ffn 



= 2^A/| [ F(/)e^"'^"' 
J— 00 



dt 



(1.7-10) 



Thus, cTn is proportional to v/T. 

The probability density function P(ai , • ■ ■ 0^ , 61 , ■ ■ ■ i.v) for the 2N co- 
efficients, ai , • ■ ■ ay , bi , • ■ ■ bfii may be derived in much the same fashion 
as was the probabiUty density of the noise current in section 1.4. Here N 
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is arbitrary but fixed. The expression analogous to (1.4-5) is the 2N fold 
integral 

P(a, , ■ ■ . , ftv) = (2^)-^'^ [^ du,--- £" dv, (1.7-11) 

exp {—iiaiUi + ■ • ■ + hsVs) — cT + vTE\ 
where 

E = — r dd exp \i E (un C„ + p„ 5„) cos nd + (r„ Cn - «n 5„) sin nO 
Ztt Jo L "-1 -• 

(1.7-12) 

in which C„ - i5„ is defined as the Fourier transform (1.7-4) of F(t). 

The next step is to show that (1.7-U) approaches a normal law in 2N di- 
mensions as v -» 00 . This appears to be quite involved. It will be noted 
that the integrand in the integral defining E is composed of N factors of the 
form 
exp [ipn cos {nO — i/'„)] 

= MPn) + 2i cos {ne - ^rdJiM - 2 cos {2ne - 2^„)72(p„) H 

where 

pl = (ul + vlXCl + Si) = ^^aliiil + vl). 

As r becomes large, it turns out that the integral (1.7-11) for the prob- 
ability density obtains most of its contributions from small values of u and v. 
By substituting the product of the Bessel function series in the integral for 
E and integrating we find 






£ = n UPn) + ^ + 5 + C 



where A is the sum of products such as 

-2i cos {^k^( - ^t - H)Jiipk)Ji(pdJi(pk+t) times N - 3 A's 
in which < k < Undl < k -\- 1 < N. Similarly B is the sum of products 
of the form 

-2i cos {il^2k - 2\l/k)Ji(pik)J2{pk) times iV - 2 /o's 

C consists of terms which give fourth and higher powers in w and v. There 
are roughly iVV^ terms of form A and N/2 terms of form B. 

Expanding the Bessel functions, neglecting all powers above the third and 



---w"4*;- '-^' 
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proceeding as in section 1.4, will give us the normal distribution plus the first 
correction term. It is rather a messy affair. An idea of how it looks may- 
be obtained by taking the special case in which F{i) is an even function cf / 
and neglecting terms of type B. Then 

P(oi , ■ ■ ■ a^ , ^ , • • ■ i;v) = (1 + 7/) n T- (1.7-12) 



2x0- „ 



where 






■n = {2vT)-"^ S [xk+t{xkxt - yuyt) + 2 ykUynyt] (1.7-13) i 

k.t 

and the summation extends over 2 < k ■}- I < N with k < I. 

It is seen that if T and N are held constant, the correction term i; ap- ^j 

proaches zero as v becomes very targe. A very rough idea of the magnitude 
of Tj may be obtained by assuming that unity is a representative value of the 
x's and y's. Further assuming that there are A'^^ terms in the summation 
each one of which may be positive or negative suggests that magnitude of 
the sum is of the order of iV. Hence we might expect to find that tj is of 
the order of N(2vTy^'\ 

PART II > 

POWER SPECTRA AND CORRELATION FUNCTIONS 

2.0 Introduction 

A theory for analyzing functions of time, t, which do not die down and j 

which remain finite as t approaches infinity has gradually been developed J 

over the last sixty years. A few words of its history together with an ex- 
tensive bibhography are given by N. Wiener in his paper on "Generalized : 
Harmonic Analysis". G. Gouy, Lord Rayleigh and A. Schuster were led 
to study this problem in their investigations of such things as white light 
and noise. Schuster invented the "periodogram" method of analysis which 
has as its object the discovery of any periodicities hidden in a continuous i 
curve representing meteorological or economic data. I 

" Ada Math., Vol. 55, pp. 117-258 (1930). See also "Harmonic Analysis of Irregular 
Motion," Jour. Math, and Phys. S (1926) pp. 99-189. , 

'*The periodogram was first introduced by Schuster in reference 10 cited In Section 
1.7. He later modified its definition in the Trans. Camb. Phil. Soc. 18 (1903), pp. 107- 
135, and still later redefined it in "The Periodogram and its Optical Analogy," Proc. Roy. 
Soc, London, Ser. A, 77 (1906) pp. 136-140. In its final form the periodogram is equiva- 
lent to \w{J), where w(j) is the power spectrum defined in Section 2.1, plotted as a func- 
tion of the period T = {2irf)-K 
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The correlation function, which turns cut to be a very useful tool, appar- 
ently was introduced by G. I. Taylor.^' Recently it has been used by quite 
a few writers" in the mathematical theory of turbulence. 

In section 2.1 the power spectrum and correlation function of a specific 
function, such as one given by a curve extending to t = « , are defined by 
equations (2.1-3) and (2.1^) respectively. That they are related by the 
Fourier inversion formulae (2.1-5) and (2.1-6) is merely stated; the dis- 
cussion of the method of proof being delayed until sections 2.3 and 2.4. In 
section 2.3 a discussion based on Fourier series is given and in section 2.4 a 
parallel treatment starting with Parseval's integral theorem is set forth. 
The results as given in section 2.1 have to be supplemented when the func- 
tion being analyzed contains a d.c. or periodic components. This is taken 
up in section 2.2. 

The first four sections deal with the analysis of a specific function of /. 
However, most of the appHcations are made to functions which behave as 
though tliey are more or less random in character. In the mathematical 
analysis this randomness is introduced by assuming the function of t to be 
also a function of suitable parameters, and then letting these parameters be 
random variables. This question is taken up in section 2.5. In section 2.6 
the results of 2.5 are applied to determine the average power spectrum and 
the average correlation function of the shot effect current. The same thing 
is done in 2.7 for a flat top wave, the tops (and bottoms) being of random 
length. The case in which the intervals are of equal length but the sign 
of the wave is random is also discussed in 2.7. The representation of the 
noise current as a trigonometrical series with random variable coefficients 
is taken up in 2.8. The last two sections 2.9 and 2.10 are devoted to prob- 
ability theory. The normal law and the central hmit theorem, respectively, 
are discussed. 

2.1 Some Results of Generalized Harmonic Analysis 

We shall first state the results which we need, and then show that they are 
plausible by methods which are heuristic rather than rigorous. Suppose 
that /(/) is one of the functions mentioned above. We may think of it as 
being specified by a curve extending from / = — oo to / = «>. /(/) may 
be regarded as ccmpoEed of a great number of sinusoidal components whose 
frequencies range from to + «. It does not necessarily have to be a noise 
current, but if we think of it as such, then, in flowing through a resistance of 
one ohm it will dissipate a certain average amount of power, say p watts. 

" Diffusion by Continuous Movements, Proc. Loud. Math. Soc, Ser. 2, 20, pp. 196- 
212 (1920). 

" See the text "Modern Developments in Fluid Dynamics" edited by S. Goldstein, 
Oxford (1938). 
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That portion of p arising from the components having frequencies between 
/and/+ (// will be denoted by w(f)dfy and consequently 



= fw(f)df (2.1-1) 



Since w(f) is the spectrum of the average power we shall call it the "power 
spectrum" of /(/). It has the dimensions of energy and on this account is 
frequently called the "energy-frequency spectrum" of 7(0. A mathematical 
formulation of this discussion leads to a clear cut definition of w(J). 

Let $(/) be a function of /, which is zero outside the interval <t < T and 
is equal to I(t) inside the interval. Its spectrum S(f) is given by 

Sif) = ( Iii)e-"'^'dt (2.1-2) 

Jo 

The spectrum of the power, w{f), is defined as 

w{f) = Limit ?i^^' (2.1-3) 

where we consider only values of / > and assume that this limit exists. 
This is substantially the definition of w(f) given by J. R. Carson and is 
useful when 7(0 has no periodic terras and no d.c. component. In the 
latter case (2.1-3) must either be supplemented by additional definitions or 
else a somewhat different method of approach used. These questions will 
be discussed in section 2,2. 

The correlation function ^(t) of /(/) is defined by the limit 

^(r) = Lin:it ^ [ I{t)I{t + t) di (2.1-4) 

r—ao T Jo 

which is assumed to exist, ^(r) is closely related to the correlation coefli- 
cients used in statistical theory to measure the correlation of two random 
variables. In the present case the value of 7(0 at time / is one variable and 
its value at a different time i + t is the other variable. 

The spectrum of the power wif) and the correlation function iP{t) are 
related by the equations 

Ml) = 4 /" .^(t) cos lirfrdT (2.1-5) 

Jo 

^(r) = f w{f) cos lirfr df (2.1-6) 

Jo 

1^ "The Statistical Energy-Frequency Spectrum of Random Disturbances," B.S.TJ., 
Vol. 10, pp. 374-381 (1931). 



■1 
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It is seen that i/'(t) is an even function of t and that 

^{0) = p (2.1-7) 

When either ^(r) or w(f) is known the other may be obtained provided the 
corresponding integral converges. 

2.2 Power Spectrum for D.C. and Periodic Components 

As mentioned in section 2.1, when 7(0 has a d.c. or a periodic com- 
ponent the limit in the definition (2.1-3) for w(f) does not exist for /equal 
to zero or to the frequency of the periodic component. Perhaps the most 
satisfactory method of overcoming this difficulty, from the mathematical 
point of view, is to deal with the integral of the power spectrum.^" 

'' wis) dg (2.2-1) 



/ 



Jo 

instead of with w(f) itself. 
The definition (2.1-t) for i^(r) still holds. If, for example, 

I{i) = A+Ccos (2Tfd - ip) (2.2-2) 

^(t) as given by (2.1^) is 

^(t) = A^ -\- - cos 27r/or (2.2-3) 

The inversion formulas (2.1-5) and (2.1-6) give 

r^ . ^ J 2 r , , , sin IitJt , 
\ w{g) dg = - \ H-r) dr 

Jo IT Jll T 

/ -1 (2.2-4) 

./'(r) = jj cos27r/rrf|jf w{g)dg^ 

" This is done by Wiener," loc. cit., and by G. W. Kenrick, "The Analysis of Irregular 
Motions with Applications to the Energy Frequency Spectrum of Static and of Telegraph 
Signals," Phil. Mag., Ser. 7, Vol. 7, pp. 176-196 (Jan. 1929). Kenrick appears to be one 
of the first to apply, to noise problem5, the correlation function msthod of computing the 
power spectrum (one of his problems is discussed in Ssc, 2.7). He bases his work on re- 
sults due to Wiener. Khintchine, in "Korrelationstheorie der stationiren stochastischen 
Prozesse," Math. A?malen, 109 (1934), pp. 604^615, proves the following theorem; A 
necessary and sufficient condition that a function R{t) may be the correlation function of 
a continuous, stationary, stochastic process is that Rit) may be expressed as 

R(t) = j cos tx dF{x) 

where Fix) is a certain distribution function. This expression for Rit) is essentially the 
second of equations (2.2-4). Kliintchine's work has hsm esteniei by H. Cran.T, "On 
the theory of stationary random processes," Ann. of Math., Ser. 2, Vol. 41 (19tJ), pp. 
215-230, However, Khintchine and Cramir appear to be interested pri-narily in ques- 
tions of existence, representation, etc., and do not stress the concept of the power spectrum. 
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where the last integral is to be regarded as a Stieltjes' integral. When the 
expression (2.2-3) for i^(t) is placed in the first formula of (2,2-4) we get 

A- when < / < /o 



/ w(^) dg = 
Jo 



When this expression is used in the second formula of (2.2-4), the increments 
of the differential are seen to be .4^ at/ = and C'^/l at/ = /o . The re- 
sulting expression for ^(t) agrees with the original one. 

Here we desire to use a less rigorous, but more convenient, method of 
dealing with periodic components. By examining the integral of w(f) as 
given by (2.2-5) we are led to write 

w(f) - 2A'5(f) + y «(/ - /o) ' (2.2-6) 

where S(x) is an even unit impulse function so that if e > 

[ S{x) dx = i f 5{x) dx= i (2.2-7) 

Jo 2 J-t 

and 8{x) = except at x = 0, where it is infinite. This enables us to use 
the simpler inversion formulas of section 2.1. The second of these, (2.1-6), 
is immediately seen to give the correct expression for 1^(7-). The first one, 
(2.1-5), gives the correct expression for w(f) provided we interpret the in- 
tegrals as follows: 

j cos lirfrdT = |6f/) 

(2.2-8) 

/ cos 2Tr/oT cos 27r/T dr = |5(/ — /o) 
Jo 

It is not hard to show that these are in agreement with the fundamental 
interpretation 

f " fi-"'^' dt^f e''"^' dt = b{j) (2.2-9) 

J^o J— at 

which in its turn follows from a formal application of the Fourier integral 
formula and 

[ 6(/)e""'^'(f/ = I 5(/)e-"'^'rf/ = 1 (2.2-10) 

We must remember that /o > and/ > in (2.2-8) so that «(/ + /o) = 
for/ > 0. 
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The definition (2.1-3) for w{f) gives the continuous part of the power 
spectrum. In order to get the part due to the d.c. and periodic com- 
ponents, which is exemplified by the expression (2.2-6) for w(f) involving 
the 6 functions, we must supplement (2.1-3) by adding terms of the type 



2A'S{f) + ^j S{f - /o) = [Limit 2-1^'] 8(J) 



(2.2-11) 



The correctness of this expression may be verified by calculating S(f) for 
the /(/) of this section given by (2.2-2), and actually carrying out the limiting 
process. 

2.3 Discussion of Results of Section One — Fourier Series 

The fact that the spectrum of power w(f) and the correlation function 
1^(t) are related by Fourier inversion formulas is closely connected with 
Parseval's theorems for Fourier series and integrals. In this section we shall 
not use Parseval's theorems explicitly. We start with Fourier's series and 
use the concept of each component dissipating its share of energy inde- 
pendently of the behavior of the other components. 

Let that portion of /(/) which Ues in the interval < / < T be expanded 
in the Fourier series 

m = f + t («.. CCS ?^' + K sin "p) (2.3-1) 

where 

dt 



«. = y. j^ m cos -^ 



(2.3-2) 



Then for the interval —T<t<T—T, 



7/, , ^ flo . V /^ 27rK(/ + t) , , . 2wn (t + t) \ 

7(i + t) = - + Z^ ( a„ cos + bn sin 1 (2.3-3) 

Multiplying the series for 7(0 and /(/ + t) together and integrating with 
respect to / gives, after some reduction, 

^}J mi{i + T)di 

= 1+ i;J(^; + ^^o cos ^-1?. 4.0(f) '''"'' 



■.. ^ ...--.■' ■*■ " .■ 
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where the last term represents correction terms which must be added be- 
cause the series (2.3-3) does not represent /(/ + t) in the interval (T — r, T) 
when T > 0, or in the interval (0, — r) if t < 0. 

If lip) were a current and if it were to flow through one ohm for the in- 
terval (0, T)j each component would dissipate a certain average amount 
of power. The average power dissipated by the component of frequency 
y„ = n/T cycles per second would be, from the Fourier series and elementary 
principles, 

i (A + 6") w-atts, « ?^ 

(2.3-5) 
-^ watts, n = 

The band width associated with the wth component is the difference in 
frequency between the « + 1 th and nth components : 

/ ,_«+! »_!. 

Hence if the average power in the band f,f-\- df is defined as w(f)df, the 
average power in the band /„+i — /„ is 



w( 



'(/n)C/n+l - /n) = W ( ^j J, 

and, from (2.3-5), this is given by 

7.(0) i=|. «=0 



(2.3-6) 



When the coefficients in (2.3-4) are replaced by their expressions in terms 
of w(/) we get 



lj[' /(/)/(/ + r)rf/ + o(?^) 

= / w(/) cos 27r/T df 
Jo 



cos -^ 

lirtiT dn 
^.cos — - 



(2.3-7) 
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where we have assumed T so large and w{f) of such a nature that the summa- 
tion may be replaced by integration. 

If / remains finite, then as T -> « with t held fixed, the correction term 
on the left becomes negligibly small and we have, upon using the definitions 
(2.1-4) for the correlation function rpir), the second of the fundamental 
inversion formulas (2.1-6). The first inversion formula may be obtained 
from this at once by using Fourier's double integral for w(f). 

Incidentally, the relation (2.3-6) between w(J) and the coefficients a„ and 
bn is in agreement with the definition (2.1-3) for w(f) as a fimit involving 
I S{f) 1 1 From the expressions (2.3-2) for o„ and K , the spectrum S(Jn) 
given by (2.1-2) is 

Then, from (2.1-3) w(/„) is given by the limit, as T~^ «, of 



© ^' 



and this is the expression for w ( - 1 given by (2.3-6). 

2.4 Discussion of Results of Section One— Parseval's Theorem 

The use of Parseval's theorem" enables us to derive the results of section 
2.1 more directly than the method of the preceding section. This theorem 
states that 

[^ F,U)F^{J) df=[ G:(/)G2(-/) dt (2.4-1) 

J— to J—ro 

where Fi, d and F^, G2 are Fourier mates related by 

.+=0 



FU) = I G{t)e 
Gil) = I 



^"^ F(f)e''"' df 



(2.4-2) 



It may be proved in a formal manner by replacing the Fi on the left of 
(2.4-1) by its expression as an integral involving Gi(t). Interchanging the 

" E. C. Titchmarsh, introduction to the Theory of Fourier Integrals, Oxford (1937). 
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order of integration and using the second of (2.4-2) to replace F2 by G2 gives 
the right hand side. 

We now set G\{t) and G^it) equal to zero except for intervals of length T. 
These intervals and the corresponding values of Gi and G2 are 

Gi(0 = /{/), < ^ < r (2.4-3) 

G^it) = I{- t-\- t), T - T <t < T 

From (2.4 3) it follows that Fi(f) is the spectrum S(f) of /(/) given by equa- 
tion (2.1-2). Since I(t) is real it follows from the first of equations (2.4-2) 
that 

■ Si-f) = 5*(/), (2.4-4) 

where the star denotes conjugate complex, and hence that | S{f) \ is an 
even function of/. 
The first of equations (2.4-2) also gives 



(2.4-5) 



Jj-T 

Jo 

When these G's and F's are placed in (2.4-1) we obtain 

[ I 5(/) I 'e-''^' dj = I ^ 1(1)1(1 + t) di (2.4-6) 

where we have made use of the fact that G2(—t) is zero except in the inter- 
val — T<t < T — T and have assumed t > 0. If r < the hmits of 
integration on the right would be — t and T. 

Since [ S(f) f is an even function of / we may write (2.4-6) as 

, ^ jj li'mt ^r)dt + (^-pj = jf 2M?ir COS lirfr df (2.4-7) 

If we now define the correlation function ^(t) as the hmit, as T—* oc ^ of the 
left hand side and define w(f) as the function 

w(f) = Limit ^'^y^' , / > (2.1-3) 



r-»co 



T 



we obtain the second, (2.1-6), of the fundamental inversion' formulas. As 
before, the first may be obtained from Fourier's integral theorem. 



..-ill 
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In order to obtain the interpretation of w(f)df as the average power dis- 
sipated in one ohm by those components of I{l) which lie in the band /, 
/ + df, we set T = in (2.4-7) : 

Limit i r fit) dt = I w{f) dj (2.4-8) 

r-.M 1 Jo Jo 

The expression on the left is certainly the total average power which would 
be dissipated in one ohm and the right hand side represents a summation 
over all frequencies extending from to «: . It is natural therefore to in- 
terpret w(/)rf/ as the power due to the components in/,/+ df. 

The preceding sections have dealt with the power spectrum w(f) and corre- 
lation function i/'(t) of a very general type of function. It will be noted 
that a knowledge of iv{f) does not enable us to determine the original func- 
tion. In obtaining Mf), as may be seen from the definition (2.1-3) or from 
(2.3-6), the information carried by the phase angles of the various compo- 
nents of /(/) has been dropped out. In fact, as we may see from the Fourier 
scries representation (2.3-1) of /(/) and from (2.3-6), it is possible to obtain 
an infinite number of different functions all of which have the same w(f), 
and hence the same ^(t). All we have to do is to assign different sets of 
values to the phase angles of the various components, thereby keeping 
(In + Jn constant. 

2.5 Harmonic Analysis for Random FrNCiioNs 

In many applications of the theory discussed in the foregoing sections 
I{t) is a function of / which has a certain amount of randomness associated 
with it. For example /(/) may be a curve representing the price of wheat 
over a long period of years, a component of air velocity behind a grid placed 
in a wind tunnel, or, of primary interest here, a noise current. 

In some mathematical work this randomness is introduced by considering 
/(/) to involve a number of parameters, and then taking the parameters to 
be random variables. Thus, in the shot effect the arrival times /i ,t2 , ■ ■ ■ Ik 
of the electrons were taken to be the parameters and each was assumed to be 
uniformly distributed over an interval (0, T). 

For any particular set of values of the parameters, /(/) has a definite power 
spectrum w(f) and correlation function i/'(t). However, now the principal 
interest is not in these particular functions, but in functions which give the 
average values of w(f) and ^(r) for fixed /and r. These functions are ob- 
tained by averaging w(f) and )/'(t) over ihe ranges of the parameters, using, 
of course, the distribution functions of the parameters. 

By averaging both sides of the appropriate equations in sections 2.1 and 
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2.2 it is seen that our fundamental inversion formulae (2.1-5) and (2.1t6)^ 
are unchanged. Thus, -; 



«'(/)= 4 f ^(t) cos2jr/T(fr (2.5-1) 

Jo 

^(t) = [ w{/) COS iTrfrdf (2.5-2) 

Jo 

where the bars indicate averages taken over the parameters with /or rheld 
constant. 

The definitions of w and i^ appearing in these equations are Hkewise ob- 
tained from (2.1-3) and (2.1-4) 

MI) = Limit ^ly (2.5-3) 

and 

^(t) = Limit i f I{l)I{t + T)dt (2.5-4) 

T—oo -I JO 

The values of f and t are held fixed while averaging over the parameters. 
In (2.5-3) S(^) is regarded as a function of the parameters obtained from 
m by 

S{J) = r Iit)e~'"'' dt (2.1-2) 

Jo 

Similar expressions may be obtained for the average power spectrum for 
d.c. and periodic components. All we need to do is to average the ex- 
pression (2.2-11) 

Sometimes the average value of the product T{l)l(i + t) in the definition 
(2.5-4) of 1^(7-) is independent of the lime T. This enables us to perform 
the integration at once and obtain 

Hr) = I{i)I{t + t) (2.5-5) 

This introduces a considerable simplification and it appears that the simplest 
method of computing wf) for an 7(0 of this sort is first to compute ^{t), and 
then use the inversion formula (2.5-1). 

2.6 First Example--The Shot Effect 

We first compute the average on the right of (2.5-5). By using the 
method of averaging employed many times in part I, we have 

mi{t + r) = E p{K) U{i)U{t + r) (2.6-1) 



MATHEMATICAL ANALYSIS OF RANDOM NOISE 321 

where p(K) is the probability of exactly K electrons arriving in the inter- 
val (0, D, 

PiK) = ^-^ e-'^ (1.1-3) 

and 

IkO) =JlFit- k) (1.3-1) 

Multiplying /«(/) and lK{t + t) together and averaging ii , 12, ■ ■ ■ Ik 
over their ranges gives 

I,it)I,{t + r) = tt r^--- f^-^Hi-k)Fii-hr-Q 

i-l in=I Jo I Jq I 

This is similar to the expression for Jl(t) which was used in section 1.3 to 
prove Campbell's theorem and may be treated in much the same way. 
Thus, if t and / + t lie between A and T — A, the expression above becomes 



K 



£" F(l)F(l -\-r)dt + ^^^;^ [£" F(0 dl'J 



When this is placed in (2.6-1) and the summation performed we obtain 
an expression independent of T. Consequently we may use (2.5-5) and get 

Ut) = V f Fit)F(t + r) rf/ + I{Ff (2.6-2) 

J— 00 

where we have used the e.xpression for the average current 

/+" 
F(l) dt (1.3-4) 

00 

In order to compute w{f) from ^(7) it is convenient to make use of the 
fact that V'(t) is always an even function of t and hence (2.5-1) may also 
be ™tten as 



(/) = 2 / i/'(t) cos lirfr dr (2.6-3) 

•i—tf) 



Then 



dt F{1) / dr F{1 + r) COS 2x/t 

/■+- „ 

V 2 / /(/)" cos 2f/t dr 

J-ao 



■.*««rt-"-'- '■■^. -=.^-'vi^-.- /-..,.->,*.'».■. -.^S.';-: -^^,.,,--fc;. :. l.■ll■.^^. ,-,4. 
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= 21- Real Part of f di F{t)e-^'"" T dt' F{ty^'"' -'- 

J— so J— M ' 

'f—tc 

= 2H^(/)l' + 2/(^'5(/) (2.6-4) 

In going from the first equation to the second we have written /' = / + t 
and have considered cos Itt/t to be the real part of the corresponding ex- 
ponential. In going from the second equation to the third we have set 

s(f) = [ F{t)e-^"^' dl (2.6-5) 



and have used 

e'-'^Ut = h{f) (2.2-9) 



£ 



The term in w{J) involving 5(/) represents the average power which would 
be dissipated by the d.c. component of I{l) in flowing through one ohm. 
It is in agreement with the concept that the average power in the band 
0</<e, €>0 but very small, is 

fw{f)dt= iTiff ( mdf 

•'0 Ja (2.6-6) 

The expression (2.6-4) for wij) may also be obtained from the definition 
(2.5-3) for 'zt'(/)plus the additional term due to the d.c. component ob- 
tained by averaging the expressions (2.2-11). We leave this as an exercise 
for the reader. He will find it interesting to study the steps in Carson's^^ 
paper leading up to equation (8). Carson's R{<ji) is related to our w{f) by 

wij) = 27rJ2(oj) 

and his /(/w) is equal to our s(f). 

Integrating both sides of (2.6-4) with respect to/ from to «; and using 



gives the result 



'^ Loc. cit. 



P = [ Mf) df 
P-f = 2v\'^ \s{f)Uf (2.6-7) 

Jo 
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I 

This may be obtained immediately from Campbell's theorem by applying 
Parseval's theorem. 

As an example of the use of these formulas we derive the power spectrum 
of the voltage across a resistance R when a current consisting of a great num- 
ber of very short pulses per second flows through R. Let F{t — h) be the 
voltage produced by the pulse occurring at time tk . Then 

where <p(t) is the current in the pulse. We confine our interest to relatively 
low frequencies such fhat we may make the approximation 

J— 00 

^(0 dt = Rq 






where q is the charge carried through the resistance by one pulse. From 
(2.6-4) it follows that for these low frequencies the continuous portion of 
the power spectrum for the voltage is constant and equal to 

w{J) = 2vR'i = 2IR\ (2.6-8) 

where I = vq\s the average current flowing through R. This result is often 
used in connection with the shot effect in diodes. 

In the study of the shot effect it was assumed that the probability of an 
event (electron arriving at the anode) happening in dt was vdt where v is the 
expected number of events per second. This probability is independent of 
the time /. Sometimes we wish to introduce dependency on time.'^ As an 
example, consider a long interval extending from to T. Let the prob- 
ability of an event happening in t,t -\- di be Kp(t)dt where K is the average 
number of events during T and p(t) is a given function of t such that 

• r 

p{t)dr= 1 



/ 

Jo 



For the shot effect p{i) = 1/T. 

What is the probability that exactly K events happen in T? As in the 
case of the shot effect, section 1.1, we may divide (0, T) into A^ intervals 
each of length At so that NAt = T. The probability of no event happening 
in the first At is 

1 - Kp{^)At 



-"(!) 



18 A careful discussion of this subject is given by Hurwitz and Kac in "Statistical 
Analysis of Certain Types of Random Functions." I understand that this paper will 
soon appear in the Annals of Math. Statistics. 



iViiiiiJi^ Vmiiiiiiit I'wi i'l'V ^^^< ^^mim^=kiipai^- 
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The product of N such probabilities is, as A^ — » «= , A/ ^ 0; 

expF-^l p{t)dt\ = e^" 

This is the probability that exactly events happen in T. In the same way 
we are led to the expression 



r-K 



^ B-" (2.6-9) 

K. ! 

for the probability that exactly K events happen in f. 

When we consider many intervals (0, T) we obtain many values of K and 
also many values of / measured / seconds from the beginning of each interval. 
These values of / define the distribution of / at time i. By proceeding as in 
section 1.4 we find that the probability density of / is 

P(/, ,) = 1 j^ du exp [-»■«/ + ^ j[' PixW"''-" - 1) d^ 
The corresponding average and variance is 

1 = K ( p{x)F{t - x) dx 

Jo 

(/ - 7)' = K f p{x)F\t - x) dx (2.6-10) 

Jo 

If S{j) is given by (2.1-2) and s{J) by (2.6-5) (assuming the duration of 
F{t) short in comparison with T) the average value of | S{}) \ may be ob- 
tained by putting (1.3-1) in (2.1-2) to get 



5.(/)=.(y)L«-^"""* 



Expressing Sk{j) 5*(0, where the star denotes conjugate complex, as a 
double sum and averaging over the /jt's, using p{t), and then averaging over 
the iC's gives 

FW)? = K. I 5(/) 1= [l + ^ I jf"" p{x)e-'''^' dx I'] (2.6-11) 

This may be used to compute the power spectrum from (2.5-3) provided 
p{x) is not periodic. If p{x) is periodic then the method of section 2.2 
should be used at the harmonic frequencies. If the fluctuations of p{t) are 
slow in comparison with the fiuctuations of F{t) the second term within the 
brackets of (2.6-11) may generally be neglected since there are no values of 
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/which make both it and s(f) large at the same time. On the other hand, 
if both ^(0 and F{t) fluctuate at about the same rate this term must be 
considered. 

2.7 Second Example— Random Telegraph Signal" 

Let T(t) be equal to either <z or — o so that it is of the form of a flat top 
wave. Let the intervals between changes of sign, i.e. the lengths of the 
tops and bottoms, be distributed exponentially. We are led to this dis- 
tribution by assuming that, if on the average there are fi changes of sign per 
second, the probability of a change of sign in /, ; + dt is fidt and is independ- 
ent of what happens outside the interval /, i -\- dt. From the same sort of 
reasoning as employed in section 1.1 for the shot effect we see that the 
probability of obtaining exactly K changes of sign in the interval (0, T) is 

PiK) =^^"''" (2.7-1) 

We consider the average value of the product r(t)l{t + r). This product 
is a if the two /'s are of the same sign and is — a if they are of opposite sign. 
In the first case there are an even number, including zero, of changes of sign 
in the interval (t, t -\- r), and in the second case there are an odd number of 
changes of sign. Thus 

Average value of I{t)I{i -1- t) (2.7-2) 

= a X probabiHty of an even number of 
changes of sign in (, / + t 

— a X probability of an odd number of 
changes of sign in ^, / + t 

The length of the interval under consideration is|/+T — ^| = |t| seconds. 
Since, by assumption, the probability of a change of sign in an elementary 
interval of length A/ is independent of what happens outside that interval, 
it follows that the same is true of any interval irrespective of when it starts. 
Hence the probabilities in (2.7-2) are independent of t and may be obtained 
from (2.7-1) by setting T = \t\. Then (2.7-2) becomes, assuming r > 
for the moment, 

mm + r) = a\p{Q>) + p{2) + ^(4) + • ■ •] 

-a[p{\) +/.(3) +/.(S)+ ...I 



= a\-'' 



[-^-'#--] 



= a'e-'"' (2.7-3) 



" Kenrick, cited in Section 2.2. 



•'-•**. 
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From (2.5-5), this gives the correlation function for /(/) 

^(r) = aV'"'^' (2.7-4) 

The corresponding power spectrum is, from (2.5-1), 

w{f) = ^i \ e'^"" cos 2jr/r dr 



2aV 



(2.7-5) 



■K-^P + M^ 



Correlation functions and power spectra of this type occur quite fre- 
quently. In particular, they are of use in the study of turbulence in hydro- 
dynamics. We may also obtain them from our shot effect expressions if we 
disregard the d.c. component. All we have to do is tc assume that the 
effect F{t) of an electron arriving at the anode at time / = is zero for 
t < 0, and that F{t) decays exponentially with time after jumping to its 
maximum value at / = 0. This may be verified by substituting the value 

Fit) =2aA/~^e'"'\ i>0 (2.7-6) 



for F{t) in the expressions (2.6-2) and (2.6-4) (after using 2.6-5) for the 
correlation function and energy spectrum of the shot effect. 

The power spectrum of the current flowing through an inductance and a 
resistance in series in response to a very wide band thermal noise voltage is 
also of the form (2.7-5). 

Incidentally, this gives us an exanlple of two quite different /(O's, one a 
flat top wave and the other a shot effect current, which have the same corre- 
lation functions and power spectra, aside from the d.c. component. 

There is another type of random telegraph signal which is interesting to 
analyze. The time scale is divided into intervals of equal length h. In an 
interval selected at random the value of I{t) is independent of the values in 
the other intervals, and is equally likely to be +fl or —a. We could con- 
struct such a wave by flipping a penny. If heads turned up we would set 
T{t) = a'mO < t < h. H tails were obtained we would set I(t) = — a in 
this interval. Flipping again would give either -f o or —a for the second 
interval h < t < 2h, and so on. This gives us one wave. A great many 
waves may be constructed in this way and we denote averages over these 
waves, with t held constant, by bars. 

We ask for the average value of I{t)I(t + t), assuming t > 0. First 
we note that if t > A the currents correspond to different intervals for all 
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values of /. Since the values in these intervals are independent we have 



l{t)nt + r) = I{1) l{t-\-T) = 

for all values of / when t > h. 

To obtain the average when t < A we consider / to lie in the first interval 
< t < h. Since all the intervals are the same frcm a statistical point 
of view we lose no generality in doing this. If / + t < A, i.e., t < h — t, 
both currents Ue in the first interval and 



/(/)/(/ + t) = a' 

If / > A — T the current I{t + t) corresponds to the second interval and 
hence the average value is zero. 

We now return to (2.5-4). The integral there extends from to T. 
When T > h, the integrand is zero and hence 

^(r) =0, T> k (2.7-7) 

When T < h, our investigation of the interval < t < k enables us to write 
down the portion of the integral extending from to h: 

[ Iit)Iit -\- T)dt = f a^dt+ I Odi 

Jq jo Jh-T 

= a\h - t) . ' 

Over the interval of integration (0, T) we have T/k such intervals each 
contributing the same amount. Hence, from (2.5-4), 



^(r) = Limit ^.f(/^-r) 
r-»o3 I ft 



The power spectrum of this type of telegraph wave is thus 
w(f) = 4(i' [ (l -j) cos 2jr/r dr 



(2.7-8) 



(2.7-9) 



This is seen to have the same general behavior as w(f) for the first type 
of telegraph signal given by (2.7-5), when we relate the average number, 
fi, of changes of sign per second to the interval length k by iih = 1. 



i i ^M i l ii- W ji w a ^A i J i ta *. • 



""*^*^^***fcttt4iaS\ t«^i^ii»iart-.iiH!fc*i*wi».iKwt«a*.^rt../'-*--t-*.^«'-^^tt-- ■ - 
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2.8 Representation or Noise Cuebent 

In section 1.8 the Fourier series representation of the shot effect current 
was discussed. This suggests the representation* 

N 

7(0 = E («- cos 6i„/ + 6„ sin WnO (2.8-1) 

n-l 



where 



o>. = 27r/n , /„ = «A/ (2.8-2) 



an and h are taken to be independent random variables which a re distributed 
normally about zero with the standard deviation y/w{J„)LJ. w(f) is the 
power spectrum of the noise current, i.e., w{f) dfis the average power which 
would be dissipated by those components of /(/) which lie in the frequency 
range/,/ + df'ii they were to flow through a resistance of one ohm. 

The expression for the standard deviation of fl„ and b„ is obtained when 
we notice that A/ is the width of the frequency band associated with the nth 
component. Hence w(/„)A/ is the average energy which would be dissi- 
pated if the current 

a„ cos a)„/ + 6n sin &)„/ 

were to flow through a resistance of one ohm, this average being taken over 
afl possible values of a„ and b„ . Thus 



w 



(/■„)A/= alcos'unt-^- 2a„&„cos ojjsinwnf + 6„sin wj = an = h„ (2.8-3) 

The last two steps follow from the independence of o„ and J„ and the identity 
of their distributions. It will be observed that w{}), as used with the repre- 
sentation (2.8-1), is the same sort of average as was denoted in section 2.5 
by wij). However, w{j) is often given to us in order to specify the spectrum 
of a given noise. 

For example, suppose we are interested in the output of a certain filter 
when a source of thermal noise is applied to the input. Let A{f) be the 
absolute value of the ratio of the output current to the input current when a 
steady sinusoidal voltage of frequency / is applied to the input. Then 

u-U) = cA\j) (2.8-4) 

*As mentioned in section 1.7 this sort of representation was used by Einstein and 
Hopf for radiation. Shottky (1918J used (2,8-1), apparently without explicitly taking 
the coefficients to be normally distributed. Nyquist (1932) derived the normal distribu- 
tion from the shot eSect. 
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If W is the average power dissipated in one ohm by /(/), 

W = Limit ^ / f{t) dt = \ w(f) df 

= cfA\f)d/ 



(2.8-5) 



which is an equation to determine c when W and A(f) are known. 

In using the representation (2.8-1) to investigate the statistical properties 
of /(/) we first find the corresponding statistical properties of the summation 
on the right when the a's and b's are regarded as random variables distrib- 
uted as mentioned above and t is regarded as fixed. In general, the time 
t disappears in this procedure just as it did in (2.8-3). We then let JV -^ oo 
and A/— ^ so that the summations may be replaced by integrations. Fi- 
nally, the frequency range is extended to cover all frequencies from to «= . 

The usual way of looking at the representation (2.8-1) is to suppose that 
we have an oscillogram of I{t) extending from / = to / = co . This oscil- 
logram may be cut up into strips of length T. A Fourier analysis of I(t) 
for each strip will give a set of coefiBcients. These coefficients will vary 
from strip to strip. Our representation (TAf = 1) assumes that this varia- 
tion is governed by a normal distribution. Our process for finding sta- 
tistical properties by regarding the a's and 6's as random variables while / 
is kept fixed corresponds to examining the noise current at a great many 
instants. Corresponding to each strip there is an instant, and this instant 
occurs at / (this is the I in (2.8-1)) seconds from the beginning of the strip. 
This is somewhat like examining the noise current at a great number of 
instants selected at random. 

Although (2.8-1) is the representation which is suggested by the shot 
effect and similar phenomena, it is not the only representation, nor is it 
always the most convenient. Another representation which leads to the 
same results when the limits are taken is" 

N 

/(/) = T. ^n COS (u„/ - <p„) (2.8-6) 

n~l 

where tpi ,if2 , • • • ^Pn are angles distributed at random over the range (0, 2jr) 
and 

c„ = [27X'(^)A/]"^ co„ = 27r/„ , /„ = nA/ (2.8-7) 

1" This representation has often been used by W. R. Bennett in unpublished memoranda 
written in the 1930's. 



i^-iirf' ^^i<^.^\J>^i^~'»■f'■•^'■■i■■-:''''•^^.*-i'<^^l*<!'^i^^*m^ ■ 
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In this representation I(t) is regarded as the sura of a number of sinusoidal 
components with fixed amplitudes but random phase angles. 

That the two different representations (2.8-1) and (2.8-6) of 7(0 lead 
to the same statistical properties is a consequence of the fact that they are 
always used in such a way that the "central limit theorem*" may be used 
in both cases. 

This theorem states that under certain general conditions, the distribu- 
tion of the sum of A' random vectors approaches a normal law (it may be 
normal in several dimensions**) as JV — > x . In fact from this theorem it 
appears that a representation such as 



/(/) = £ {a„ cos a)„/ + 6„ sin u„0 



(2.8-6) 



n-l 



where a„ and 6„ are independent random variables which take only the values 
± [w{f„)Aff'^, the probabiUty of each value being ^, will lead in the limit 
to the same statistical properties of /(/) as do (2.8-1) and (2.8-6). 

2.9 The Normal Distribution in Several Variables^" 

Consider a random vector r in K dimensions. The distribution of this 
vector may be specified by stating the distribution of the K components, 
xi , X2 , ■ ■ • Xk , oi r. r is said to be normally distributed when the prob- 
ability density function of the x's is of the form 

(27r)-^'' I U r'" exp [-Ix'M-'x] (2.9-1) 

where the exponent is a quadratic form in the s's. The square matrix M 
is composed of the second moments of the a-'s. 



M = 



(2.9-2) 



fin fin •• • MiK 
_filK: ■ • • y-KK_ 
where the second moments are defined by 

Mu = a-1 , {112 = a-i.r2 , etc. (2.9-3) 

I M I represents the determinant of M and x' is the row matrix 

x' = [.vi,.r2,---.rAl (2.9-4) 

X is the column matrix obtained by transposing x'. 

*See section 2,10. 
" See section 2.9, 

" H. Cramer, "Random Variables and Probability Distributions," Chap. X., Cambridge 
Tract No. 36 (1937). 



.^ 
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The exponent in the expression (2.9-1) for the probability density may 
be written out by using 

x'M-'x^i:j:f^,XrX. (2.9-5) 

where Mra is the cofactor of Urs in M. 

Sometimes there are linear relations between the a;'s so that the random 
vector r is restricted to a space of less than K dimensions. In this case the 
appropriate form for the density function may be obtained by considering 
a sequence of X-dimensional distributions which approach the one being 
investigated. 

If fi and f2 are two normally distributed random vectors their sum n + r^ 
is also normally distributed. It follows that the sum of any number of 
normally distributed random vectors is normally distributed. 

The characteristic function of the normal distribution is 



tLVC. C — 



exp -^ Z E M..2rS« (2.9-6) 



2.10 Central Limit Tiieorem 

The central limit theorem in probability states that the distribution of the 
sum of N independent random vectors ri + r2 + ■ • ■ -|- r^ approaches a 
normal law as A'^ ^ «> when the distributions of n , ra , ■ ■ ■ r^ satisfy certain 
general conditions.' 

As an example we take the case in which ri , ra , • • • are two-dimensional 
vectors^\ the components of r™ being Xn and Vn . Without loss of generality 
we assume that 

Xn = 0, Sn = 0. 

The components of the resultant vector are 

X = xi + X2 + ■ ■ ■ -[■ Xn 

(2.10-1") 
; Y = yi-\- y2-\- ■■■ -\- yN 

and, since n , ^2 , ■ ■ ■ are independent vectors, the second moments of the 

resultant are 

fia= X = a:i -f a-2 + • • • + av 

1112 = Y' = yl + yl-\- •■■ +yl (2.10-2) 

fiu = XY = xiyi + .T23'2 + ■ • • + -r.v3V 

' Incidentally, von Laue (see references in section 1.7) used this theorem in discussing 
the normal distribution of the coefficients in a Fourier series used to represent black-body 
radiation. He ascribed it to Markoff. 

" This case is discussed by J. V. Uspensky, "Introduction to Mathematical Probabil- 
ity", McGraw-Hill (1937) Chap. XV. 
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Apparently there are several types of conditions which are sufficient to 
ensure that the distribution of the residtant approaches a normal law. One 
sufficient condition is that 



JV 

J/2 
Mil 



-"' E K 



";' (2.10-3) 

The central limit theorem tells us that the distribution of the random 
vector (X, Y) approaches a normal law d.s N -^ « . The second moments 
of this distribution are given by (2.10-2). When we know the second mo- 
ments of a normal distribution we may write down the probability density 
function at once. Thus from section 2.9 



Lm12 M22J L~Atl2 mi J 

\ M \ = M11M22 ~ M12 

x' = [X, Y] 

x'M-'x = I M r'(M22X' - 2nnXY + a^uI'') 

The probability density is therefore 

(miiMzb - tJ-^r^'^ r -ti22X^ - finY^ + 2mi2XF "| ^2.10-3) 
2t L 2(Mn/i22 — /iw) J 

Incidentally, the second moments are related to the standard deviations 
ffi , 0-2 of X, Y and to the correlation coefficient t of X and Y by 

Mil = ffl , ^22 = 0-2, M12 = TO'lO'2 (2.10-4) 

and the probability density takes the standard form 

(i^^' exp r-,-^^ (^ - 2. ^^ + m (2.10-5) 
2x0-1(72 L 2(1-t)Vi fficra ff2/J 

" This is used by Uspensky, loc. cit. Another condition analogous to the Lindeberg 
condition is given by Cramerj^" loc. cit. 

(To be concluded) 



